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PREFACE 

This  report  documents  the  results  of  a  literature  review  and  three  experiments 
that  addressed  training  design  issues  relative  to  Air  Force  aircrew  training  using 
microcomputer-based  (desktop)  instructional  simulation.  This  report,  with  some  minor 
revisions,  was  previously  published  by  Dr.  Joseph  S.  Mattoon  (1994)  in  The 
International  Journal  of  Aviation  Psychology.  4(3).  The  objective  of  this  research  was 
to  examine  factors  that  influence  the  behavior,  learning,  and  performance  of  adults 
who  engage  in  training  activities  via  instructional  simulations  which  are  commonly 
used  in  Air  Force  training  programs  and  other  technical  training  environments.  The 
work  was  conducted  underwork  Unit  1123-25-15,  Unit  Level  Training  Research  and 
Applications  (ULTRA)  group.  The  work  unit  monitor  was  Dr.  Bernell  J.  Edwards,  and 
the  principle  investigator  was  Dr.  Joseph  Sterling  Mattoon. 
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DESIGNING  INSTRUCTIONAL  SIMULATIONS:  EFFECTS  OF 
INSTRUCTIONAL  CONTROL  AND  TYPE  OF  TRAINING  TASK 
ON  DEVELOPING  DISPLAY-INTERPRETATION  SKILLS 
INTRODUCTION 

Instructional  simulation  is  a  combination  of  computer-based  training  (CBT)  and 
animated  computer  graphics  that  is  used  to  teach  aircrew  and  other  technical 
personnel  how  to  perform  complex  dynamic  tasks  (Alessi,  1988).  The  power  of  these 
systems  lies  in  the  capacity  to  quickly  tailor  content  material,  practice  activities,  and 
performance  criteria  to  suit  the  learning  needs  of  each  individual,  but  research  has 
been  inconclusive  in  specifying  the  most  effective  way  to  organize  and  control  these 
variables.  There  are  two  basic  types  of  instructional  control,  learner  control  and 
program  control.  Learner-controlled  instruction  enables  individuals  to  make  their  own 
decisions  concerning  the  type  and  amount  of  material  they  receive.  In  contrast, 
program-controlled  instruction  delivers  a  predetermined  body  of  content  and  practice 
or  adjusts  these  activities  based  on  an  individual’s  performance. 

Some  researchers  have  recommended  learner  control  for  computer-based 
instruction  (Hannafin,  1984;  Milheim  &  Martin,  1991;  Tennyson,  1980,  1981),  but 
several  empirical  studies  have  often  found  no  effect  for  instructional  control  on  terminal 
performance  (Gay,  1986;  Klein,  1988;  Lahey,  Hurlock,  &  McCann,  1973)  or  concluded 
that  program  control  is  more  effective  (Avner,  Moore,  &  Smith,  1980;  Mattoon  &  Klein, 
1993;  Ross  &  Rakow,  1981).  The  majority  of  instructional  control  studies  show  that 
students  have  difficulty  managing  their  study  time,  choosing  appropriate  instructional 
activities,  and  setting  useful  performance  goals  when  they  are  given  control  over 
instruction  (Steinberg,  1977,  1989).  Some  researchers  have  found  that  learner  control 
reduces  the  time  learners  spend  on  instruction,  but  this  tends  to  hinder  their 
achievement  unless  each  individual  is  provided  with  advisement  when  choosing 
instructional  options  (Johansen  &  Tennyson,  1983;  Tennyson,  1980, 1981;  Tennyson  & 
Buttrey,  1980).  Research  has  not  resolved  the  problems  associated  with  instructional 
control  in  conventional  CBT,  and  only  a  few  studies  have  examined  this  issue  in  the 
context  of  instructional  simulation. 

Instructional  simulation  is  a  hybrid  form  of  CBT  that  has  become  an  important 
medium  for  helping  military  and  other  technical  personnel  develop  complex  skills 
(Alessi,  1988;  Gray  &  Edwards,  1991;  Mattoon  &  Thurman,  1990;  Reigeluth  & 
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Schwartz,  1989).  Complex  skills  are  quite  different  from  the  declarative  knowledge 
that  is  acquired  from  conventional  CBT.  Declarative  knowledge  is  characterized  by  a 
relatively  slow  and  deliberate  process  of  recalling  information  from  long-term  memory 
(Gagne,  1985),  whereas  complex  skills  consist  of  a  combination  of  intellectual, 
perceptual,  and  motor  abilities  and  are  associated  with  fast  and  automatic  responses 
to  stimuli  (Schneider,  1985).  Declarative  knowledge  can  usually  be  assessed  by 
discrete  responses  to  short-answer  or  multiple-choice  questions,  but  complex  skills 
are  described  by  continuous  measures  of  accuracy  and  speed. 

A  complex  task  can  be  described  as  a  number  of  subtasks  that,  when  executed 
together  or  in  a  sequence,  accomplish  a  single  goal  (Naylor,  1962).  A  longstanding 
theoretical  debate  continues  today  on  whether  to  teach  learners  how  to  perform  a 
complex  task  all  at  once  (whole-task  training)  or  break  it  down  into  constituent 
subtasks  that  are  gradually  re-integrated  into  the  whole  task  as  learning  progresses 
(part-task  training).  In  part-task  training,  each  subtask  is  described  and  practiced  as  a 
prelude  to  attempting  the  whole  task  (Gropper,  1983;  Naylor,  1962;  Stammers,  1982; 
Wightman  &  Lintern,  1985).  In  whole-task  training,  the  criterion  task  is  represented  as 
an  integrated  whole,  and  learners  practice  the  entire  task.  Part-task  training  has  been 
shown  to  reduce  some  of  the  difficulties  learners  have  when  they  first  attempt  to 
understand  and  practice  a  complex  task  (Briggs  &  Naylor,  1962;  Fabiani,  Buckley, 
Gratton,  Coles,  Donchin,  &  Logie,  1989;  Frederiksen  &  White,  1989;  Mane,  Adams,  & 
Donchin,  1989). 

Student  pilots  and  other  technical  trainees  must  learn  how  to  quickly  and 
accurately  interpret  symbolic  information  from  electronic  displays.  Instructional 
simulation  has  proven  to  be  a  valuable  tool  for  teaching  this  type  of  skill  in  Air  Force 
training  programs  (Gray  &  Edwards,  1991).  Instructional  simulations  can  teach 
aviation  students  how  to  interpret  a  variety  of  radar  and  other  dynamic  readouts  from 
multifunction  cockpit  displays. 

EXPERIMENTAL  MATERIALS  AND  PROCEDURES 

The  head-up  display  (HUD)  and  the  radar-electric  optical  (REO)  display  are 
used  for  similar  radar-tracking  tasks  in  military  aircraft  and  provide  information  that  is 
used  by  pilots  to  estimate  the  location  and  heading  of  Targets”  (aircraft  that  have  been 
detected  by  the  pilot’s  radar).  Three  experiments  examined  undergraduates’  ability  to 
learn  and  perform  target  estimations  under  different  instructional  treatments. 
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Experiment  1  compared  three  strategies  for  controlling  level  of  challenge  (task 
difficulty)  during  practice.  Experiment  2  compared  part-  and  whole-task  practice 
strategies,  and  experiment  3  examined  the  joint  effects  of  learner  and  program  control 
when  combined  with  part-  and  whole-task  training.  Subjects  estimated  targets  with  the 
HUD  in  experiments  1  and  2  and  estimated  targets  with  both  the  HUD  and  REO  in 
experiment  3. 

The  instructional  simulation  used  in  each  experiment  was  developed  with 
HyperCard  software  and  was  delivered  by  Macintosh  computers  with  12-inch 
screens.  The  en-route  and  criterion  tasks  were  executed  via  a  computer  mouse.  Each 
experiment  begin  with  a  CBT  lesson  that  taught  subjects  how  to  use  a  computer 
mouse  to  manipulate  graphic  objects  that  were  part  of  the  simulations.  The  lesson 
concluded  with  a  pretest  that  assessed  each  subject’s  initial  level  of  skill  with  the 
mouse.  The  pretest  required  the  subject  to  “drag”  a  graphic  object,  referred  to  as  the 
“locator,”  to  the  center  of  a  small  circle  of  numbers  on  the  computer  screen  and  rotate  it 
to  a  designated  number.  This  involved  the  same  basic  motor  skills  needed  to  perform 
the  criterion  tasks  in  the  experiments. 

The  instructional  simulation  contained  verbal  descriptions  and  animated 
demonstrations  of  the  target-estimation  task.  Subjects  used  a  mouse  to  position 
locator  on  the  screen  to  estimate  each  target’s  location,  and  rotated  the  locator  to 
estimate  the  target’s  heading  relative  to  the  pilot’s  aircraft  (a  stationary  symbol). 

During  practice,  a  target  symbol  appeared  after  each  estimation  to  show  the  correct 
location  and  heading.  A  target’s  angular  distance  to  the  left  or  right  of  the  pilot’s 
aircraft  was  designated  on  the  HUD  by  azimuth  (a  whole  number  that  ranged  from 
zero  to  60  degrees)  and  an  arrow  that  pointed  to  the  right  or  left.  Aspect  designated  a 
target’s  heading  and  was  represented  by  the  position  of  a  pointer  along  the 
circumference  of  a  circle  on  the  HUD.  Figure  1  shows  the  HUD,  locator,  pilot’s  aircraft, 
and  a  target  with  an  azimuth  of  45  degrees  right  and  a  left-wing  aspect  (i.e.,  left  wing  is 
pointed  toward  the  pilot’s  aircraft). 

On  each  trial,  the  target  information  was  displayed  on  the  HUD,  and  the  locator 
was  centered  above  the  pilot’s  aircraft.  The  subject’s  task  was  to  “click”  the  mouse  on 
the  locator  and  drag  it  to  the  estimated  target  location,  then  click  on  one  of  the  rotate 
buttons  to  rotate  the  locator  to  the  estimated  target  heading.  When  the  estimation  was 
complete,  the  subject  clicked  on  a  button  labeled  “done,”  and  the  program  recorded 
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Figure  1 . 

HUD,  locator,  pilot’s  aircraft,  and  target  with 
an  azimuth  of  45°  right-  and  left-wing  aspect. 


response  time  (RT)  and  accuracy  and  showed  the  target  symbol  at  the  correct  position. 
The  subjects  were  then  able  to  compare  the  position  of  the  locator  (their  estimate)  with 
that  of  the  target.  No  combinations  of  target  location  and  heading  were  repeated  on 
practice  or  posttest  items.  The  posttests  consisted  of  25  targets  to  estimate  in 
experiment  1  and  30  targets  in  experiments  2  and  3.  The  same  target  parameters 
were  used  in  each  treatment,  and  no  targets  with  the  same  location  and  heading  were 
used  more  than  once  in  practice  and  posttest  activities. 

Subjects’  mean  accuracy  (average  degree  of  error  in  target  location  and 
heading)  and  mean  RT  across  posttest  items  were  the  primary  measures  of  skill.  A 
multivariate  analysis  of  variance  (MANOVA)  was  used  for  initial  examinations  of 
subjects’  training  time  and  their  speed  and  accuracy  on  the  posttests.  Follow-up 
univariate  analyses  of  variance  (ANOVAs)  were  used  when  significant  differences 
were  detected  by  the  MANOVA.  A  significance  level  of  £  <  .05  was  used  for  all 
statistical  tests.  Mean  RT  on  the  pretest  was  used  as  a  covariate  to  partition  subjects’ 
initial  skill  for  using  a  mouse  from  the  skill  they  acquired  from  the  instructional 
simulation. 
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EXPERIMENT  1 

The  following  experiment  was  reported  previously  (Mattoon  &  Klein,  1993)  and 
is  described  here  after  a  re-analysis  of  the  data. 

Method 

Subjects.  Materials,  and  Treatments 

Subjects  were  61  undergraduates  (12  male  and  49  female)  from  a  large  public 
university  in  the  southwestern  United  States  who  participated  in  the  experiment  for 
partial  course  credit.  The  first  portion  of  the  program  provided  subjects  with  instruction 
for  using  the  HUD  to  estimate  targets.  Seventy-five  practice  trials  followed  the  HUD 
instruction.  Each  trial  included  one  target  to  estimate.  Points  for  RT  were  earned 
according  to  the  ratio-time  limit  divided  by  RT-and  accuracy  points  according  to  the 
ratio-error  limit  divided  by  degrees  of  error.  For  each  estimate  that  was  within 
accuracy  and  speed  criteria,  a  proportional  number  of  points  were  added  to  the 
subject’s  score,  and  each  estimate  that  was  below  criteria  caused  a  proportional 
number  of  points  to  be  subtracted. 

An  increase  in  speed  challenge  decreased  the  time  limit  (time  allowed  to 
estimate  a  target),  while  an  increase  in  accuracy  challenge  decreased  the  error  limit 
(degrees  of  error  allowed  in  each  estimate).  At  higher  challenge  levels,  subjects  could 
earn  more  points  for  estimates  that  met  criteria,  but  they  also  risked  losing  more  points 
for  estimates  that  were  too  slow  or  inaccurate.  Challenge  level  for  speed  and 
accuracy  were  displayed  on  two  scales  that  ranged  from  10  to  100. 

The  experimental  treatments  were  learner  control  (LC),  learner  control  with 
advisement  (LCA),  and  program  control  (PC).  Level  of  challenge  was  controlled  by 
the  subject  in  the  LC  and  LCA  groups  and  was  controlled  by  the  program  in  the  PC 
group.  The  LC  version  of  the  program  allowed  subjects  to  adjust  challenge  levels  after 
each  practice  trial  by  adjusting  a  pointer  on  each  challenge  scale.  The  LCA  version 
provided  the  same  learner  control  over  challenge  but  also  displayed  advice  on  control 
decisions  (e.g.,  “set  speed  challenge  to  35”)  as  a  function  of  the  proportion  of  points 
earned  (or  lost)  over  the  most  recent  five-trial  block.  The  PC  version  automatically 
adjusted  challenge  levels  every  five  trials  using  the  same  function  that  generated 
advice  in  the  LCA  treatment.  The  program  delivered  an  immediate  and  a  one-week 
delayed  HUD  posttest.  No  challenge  levels  were  used  on  posttests,  but  the  program 
repeatedly  prompted  subjects  to  estimate  targets  quickly  and  accurately. 
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Procedures 

The  experiment  was  a  two-variable  mixed  design  with  three  levels  of 
instructional  control  as  the  between-subjects  variable  and  test  occasion  as  the  within- 
subjects  variable.  An  approximately  equal  number  of  males  and  females  were 
assigned  to  each  treatment  in  the  order  they  arrived  at  the  computer  site.  Subjects 
completed  the  mouse  lesson  and  pretest,  HUD  instruction,  practice,  and  immediate 
posttest,  respectively.  Each  subject  returned  one  week  later  to  complete  the  delayed 
posttest. 

Results  and  Discussion 

The  covariate  (pretest  RT)  was  significantly  correlated  with  posttest  RT  (r  =  0.50, 
£<  .01)  and  error  in  estimations  of  target  heading  (r  =  -0.27,  £  <  .05)  on  the  posttests. 
Also,  error  in  target  location  was  significantly  correlated  with  error  in  heading 
(r  =  -0.29,  £  <  .05).  The  adjusted  means  for  posttest  RT  are  reported  in  Table  1 . 


Table  1.  RT  (s)  on  Immediate  and  Delayed  Posttests  Adjusted  for  the  Covariate 


Posttest 

Across 

Treatments 

Treatment 

Co  variate 

Immediate 

Delayed 

LC 

M 

12.6 

13.0 

14.6 

13.8 

SD 

4.55 

2.47 

2.23 

2.35 

LCA 

M 

11.7 

14.1 

15.8 

15.0 

SD 

4.07 

3.21 

2.27 

2.74 

PC 

M 

11.9 

14.4 

14.4 

14.4 

SD 

4.85 

3.95 

3.96 

3.96 

Across 

T  reatments 

M 

12.2 

13.8 

14.9 

14.4 

SD 

4.94 

3.21 

2.82 

3.03 

Note.  Covariate  =  pretest  RT;  LC  =  learner  control;  LCA  =  learner 
control  with  advisement;  PC  =  program  control. 
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The  initial  analysis  revealed  significant  differences  among  treatments, 
multivariate  £(4,55)  =  6.02,  £  <  .001 .  A  follow-up  ANOVA  revealed  a  treatment  by  test 
occasion  interaction  effect  on  RT,  F(2,58)  =  4.95,  £  <  .05,  and  an  analysis  of  simple 
effects  indicated  that  LC  subjects  performed  faster  on  the  immediate  posttest  (M  = 

1 3.0)  than  on  the  delayed  posttest  (M  =  1 4.6),  F(1 ,38)  =  4.64,  £  <.05.  LCA  subjects 
were  also  faster  on  the  immediate  posttest  (M  =  1 4.1 )  than  the  delayed  posttest  (M  = 
15.8),  although  this  difference  did  not  reach  the  assigned  level  of  significance,  £(1 ,38) 
=  2.81  ,£=.10.  In  contrast,  PC  subjects  performed  at  the  same  speed  on  both 
immediate  and  delayed  posttests  (M=  14.4).  This  interaction  is  illustrated  in  Figure  2. 


Posttest  Posttest 


Figure  2 

RT  on  Immediate  and  Delayed  Posttests. 

The  increase  in  RT  for  subjects  in  the  LC  and  LCA  groups  may  be  due  to  the  difficulty 
they  had  in  recalling  some  of  the  concepts  and  rules  of  the  criterion  task.  Subjects 
under  learner  control  had  to  perform  some  additional  tasks  during  practice  to  adjust 
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speed  and  accuracy  challenge  levels,  and  this  may  have  interfered  with  their  retention 
of  information.  Logan  (1985)  indicates  that  additional  subtasks  which  engage  the 
same  cognitive  resources  decrease  a  learner’s  ability  to  acquire  a  complex  skill. 
Controlling  challenge  levels  can  be  viewed  as  such  an  additional  subtask  because  it 
required  some  of  the  same  type  of  mental  calculations  as  those  used  to  estimate 
targets.  In  contrast,  the  program-control  version  of  the  instructional  simulation 
automatically  set  challenge  levels  and  probably  enabled  subjects  to  focus  more  of 
their  attention  on  the  practice  and  feedback.  This  could  be  the  reason  that  PC  subjects 
did  not  exhibit  an  increase  in  RT  across  the  retention  interval. 

Subjects  under  program  control  may  have  responded  more  quickly  on  the 
posttests  if  they  would  have  received  practice  at  lower  challenge  levels.  The  average 
challenge  level  for  PC  subjects  was  higher  than  the  average  for  LCA  subjects  on  53% 
of  the  practice  trials  and  higher  than  the  average  for  LC  subjects  on  73%  of  the  trials. 
Higher  levels  of  challenge  made  it  harder  for  subjects  to  score  points  and  made 
negative  scores  more  frequent.  The  consistently  high  challenge  level  in  the  PC  group 
may  have  induced  subjects  to  be  overly  cautious  and  take  more  time  than  they  needed 
to  estimate  targets.  Learners’  perceptions  of  their  performance  during  practice 
probably  affect  terminal  performance  and  should  be  carefully  considered  when 
instructional  treatments  are  compared. 

EXPERIMENT  2 

The  following  experiment  was  reported  previously  (Mattoon  &  Edwards,  1993) 
and  is  described  here  after  a  re-analysis  of  the  experimental  data. 

Method 

Subjects.  Materials,  and  Treatments 

Subjects  were  29  female  undergraduates  from  the  same  university  who 
participated  for  partial  course  credit.  The  program  was  modified  to  produce  two 
practice  treatments,  one  progressive  part-task  and  the  other  whole-task.  Both 
instructional  treatments  begin  with  a  CBT  lesson  that  described  and  demonstrated  the 
entire  target-estimation  task.  After  the  lesson,  the  part-task  version  presented  six 
practice  exercises  (sets  of  practice  trials).  Each  exercise  progressively  included 
practice  on  each  subtask  and  group  of  subtasks  until  the  entire  task  was  re-integrated 
and  practiced  in  the  final  exercise.  The  whole-task  version  of  the  program  required  the 
whole  task  to  be  executed  in  each  practice  trial. 
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Each  subject  controlled  the  number  of  practice  trials  presented  prior  the 
posttest.  Part-task  subjects  practiced  1/3, 1/3, 2/3, 1/3, 2/3,  and  3/3  of  the  target- 
estimation  task,  respectively,  and  were  required  to  complete  a  minimum  of  30  practice 
trials  (5  per  exercise).  An  equivalent  minimum  for  whole-task  practice  was  computed 
as  a  function  of  the  proportion  of  the  task  presented  across  the  six  part-task  exercises- 
10/3  x  5-trial  part-task  minimum  =  16.7  or  17  whole-task  trials. 

Procedures 

The  experiment  was  a  single-variable,  between-subjects  design  with  two  levels 
of  training  task  (part-  and  whole-task).  An  approximately  equal  number  of  subjects 
were  assigned  to  each  treatment  in  the  order  that  they  arrived  at  the  computer  site. 
Subjects  completed  the  mouse  lesson  and  pretest,  HUD  instruction,  practice,  and  the 
posttest,  respectively. 

Results  and  Discussion 

Significant  correlations  for  the  posttest  data  analysis  were  pretest  RT  with  target 
heading  error  (r  =  0.42,  &<  .05),  posttest  RT  with  heading  error  (r  =  -0.39,  £<  .05),  and 
target  location  error  with  heading  error  (r  =  0.42,  n<  .05).  The  initial  analysis  of  these 
measures  indicated  significant  differences  among  treatments,  multivariate  F(4,23)  = 
3.98,  ja  <  .05.  Univariate  analyses  showed  that  subjects  in  the  part-task  group 
estimated  targets  faster  (M=  16.9  s)  than  those  in  the  whole-task  group  (M  =  20.6  s), 
E(1 ,26)  =  5. 1 1 ,  ft  <  .05. 

Table  2  shows  the  means  for  number  of  practice  trials  completed,  RT  on  practice 
trials,  time  on  practice,  and  posttest  RT.  Part-task  subjects  spent  an  average  of  three 
more  minutes  on  practice  than  whole-task  subjects,  but  this  difference  was  not 
significant.  As  expected,  there  were  significant  differences  between  part-  and  whole- 
task  groups  for  RT  on  practice  trials  and  number  of  trials  completed,  multivariate 
E(3,24)  =  27.67,  ja  <  .001 .  Part-task  subjects  completed  more  practice  trials  (M  =  51 .7) 
than  whole-task  subjects  (M  =  1 7.7),  F(1 ,26)  =  48.1 0,  £  <  .001 .  Also,  RT  was  faster  for 
the  part-task  group  (M  =  17.9  seconds)  than  for  the  whole-task  group  (M  =  38.2  s), 

E(1 ,26)  =  24.29,  <  .001 . 

The  difference  in  number  of  trials  completed  and  RT  on  the  practice  exercises 
was  not  surprising  because  of  the  differences  in  minimum  trials  and  type  of  practice 
tasks  in  the  part-  and  whole-task  treatments.  However,  part-task  subjects  averaged 
72%  more  practice  trials  than  the  minimum  number  of  trials  required  by  the  program, 
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Table  2.  Number  of  Practice  Trials,  Practice  Trial  RT,  Time  on  Practice,  and  Posttest  RT 


Treatment 

Practice 

T  rials 

Practice 

Trial  RT3 

Time  on 
Practice*3 

Posttest  RT3 

Part-Task 

M 

51.7 

17.9 

14.2 

16.9 

SD 

18.63 

5.07 

4.53 

3.66 

Whole-Task 

M 

17.7 

38.2 

11.2 

20.6 

SD 

2.21 

16.26 

4.66 

4.85 

Total 

M 

36.5 

27.0 

12.9 

18.6 

SD 

22.0 

15.24 

4.76 

4.54 

aln  seconds.  ^In  minutes. 


while  whole-task  subjects  averaged  only  4%  more  than  the  minimum.  When  the  mean 
number  of  target  estimates  is  computed  according  to  the  proportion  of  the  task  that  was 
practiced  in  each  part-task  exercise,  part-task  subjects  completed  an  average  of  27.3 
target  estimates  during  practice  compared  to  an  average  of  only  17.7  by  whole-task 
subjects.  This  indicates  that  part-task  subjects  chose  to  complete  an  average  of  54% 
more  practice  than  whole-task  subjects  but  spent  an  average  of  only  27%  more  time 
than  whole-task  subjects  (a  nonsignificant  difference)  on  practice.  Evidently,  the  part- 
task  treatment  enabled  subjects  to  complete  more  practice  in  less  time  than  whole-task 
subjects,  and  this  may  account  for  part-task  subjects’  advantage  in  RT  on  the  posttest. 

EXPERIMENT  3 

The  following  experiment  was  reported  previously  in  greater  detail  (Mattoon, 
1994)  and  is  described  here  after  a  re-analysis  of  the  experimental  data. 

Method 

Subjects.  Materials,  and  Treatments 

Subjects  were  48  male  ROTC  undergraduates  from  the  same  university  who 
were  paid  $15  each  for  participating  in  the  experiment.  The  instructional  simulation 
program  was  further  modified  to  create  two  levels  of  instructional  control-learner  and 
program  control-and  two  levels  of  training  task-part-  and  whole-task.  Each  of  four 
versions  of  the  program  consisted  of  initial  instruction  on  the  HUD,  four  practice 
exercises  with  six  trials  each,  additional  instruction  on  the  HUD,  HUD  posttest, 
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overview  of  the  REO  display,  and  REO  posttest,  respectively.  The  REO  was  similar  to 
the  HUD,  but  it  designated  target  azimuth  on  a  horizontal  scale  and  designated  target 
aspect  as  a  whole  number  (0  to  180)  and  a  label-left”  for  left-wing  aspect  and  “right” 
for  right-wing  aspect. 

During  practice,  each  target  estimate  was  judged  as  correct  or  incorrect  based 
on  a  single  criterion  level  for  speed  and  for  accuracy.  The  accuracy  criterion  was  met 
by  placing  the  locator  within  7  deg  of  the  correct  target  location  and  rotating  it  to  a 
position  within  30  deg  of  the  correct  heading.  The  RT  criterion  was  set  at  a  maximum 
of  10  s  for  each  half  of  the  target-estimation  task  (estimation  of  location  and  estimation 
of  heading).  If  a  subject  estimated  50%  or  more  targets  correctly  in  an  exercise,  the 
program  encouraged  the  individual  to  go  on  to  the  next  activity  by  displaying  a 
message  on  screen-“your  performance  does  not  need  improvement.”  If  the  subject’s 
score  was  below  50%,  a  message  encouraged  the  individual  to  complete  more 
instruction-“additional  instruction  may  increase  your  accuracy  and  speed.” 

In  the  learner-control  treatment,  subjects  had  the  choice  to  complete  or  bypass 
a  segment  of  the  additional  instruction  after  each  practice  exercise.  The  program- 
control  treatment  automatically  presented  additional  instruction  if  less  than  50%  of  the 
estimates  were  correct.  Each  segment  of  additional  instruction  consisted  of  a  review  of 
the  HUD  instruction  and  strategies  for  improving  accuracy  and  speed. 

In  experiment  2,  the  part-  and  whole-task  treatments  were  distinguished  only  by 
the  type  of  practice  they  received,  but  in  experiment  3,  the  part-  and  whole-task 
treatments  also  differed  in  the  way  that  the  initial  instruction  described  and 
demonstrated  the  target-estimation  task.  The  initial  instruction  in  the  part-task 
treatment  consisted  of  four  separate  lessons.  Each  part-task  lesson  was  followed  by 
an  exercise  that  provided  six  practice  trials.  Each  exercise  required  the  subject  to 
perform  the  subtask  or  combination  of  subtasks  that  was  taught  in  the  preceding 
lesson.  The  fourth  part-task  exercise  provided  practice  on  the  whole  task. 

The  whole-task  treatment  consisted  of  the  same  amount  of  initial  instruction  and 
practice  trials  as  the  part-task  treatment.  However,  whole-task  instruction  described 
and  demonstrated  the  entire  target-estimation  task  in  one  lesson  that  was  followed  by 
four  whole-task  practice  exercises  with  six  trials  each. 

An  overview  of  the  REO  display  enabled  subjects  to  alter  target  parameters  on 
the  HUD  while  observing  corresponding  changes  in  target  information  on  the  REO. 
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The  REO  posttest  was  identical  to  the  HUD  posttest,  except  that  targets  were  estimated 
according  to  target  information  shown  on  the  REO.  Figure  3  shows  the  sequence  of 
instructional  events  for  the  part-  and  whole-task  treatments. 


Mouse  Lesson 


PART-TASK 


1 


WHOLE-TASK 


HUD  Lesson  & 

Practice  Activity  1 

NOTE:  Dashed  lines 

HUD  Lesson 

'  Additional  Instr.  1 
■ _ - _ 

1  indicate  Additional 

1 

HUD  Lesson  & 
Practice  Activity  2 


Additional  Instr.  2 


1 


I 


HUD  Lesson  & 
Practice  Activity  3 


Additional  Instr.  3 


HUD  Lesson  & 
Practice  Activity  4 


i  Additional  Instr.  4 

i _ _ _ _ 


be  presented  if  a 
subject  chose  to  do 
it  under  learner 
control,  or  if  a 
subject's  practice 
score  was  below 
criterion  under 
program  control. 


J 


Practice  Activity  1 


Additional  Instr.  1  1 

I 


Practice  Activity  2 


Additional  Instr.  2  ! 


I 


Practice  Activity  3 


Additional  Instr.  3  \ 

1 


Figure  3 

Sequence  of  Instructional  Events  for  Part-  and  Whole-Task  Treatments. 
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Procedures 

The  instructional  control  and  training  task  variables  were  completely  crossed 
and  manipulated  between  subjects  to  produce  a  2  x  2  factorial  design.  An  equal 
number  of  subjects  were  assigned  to  each  treatment  in  the  order  they  arrived  at  the 
computer  site.  They  completed  the  mouse  lesson  and  pretest,  HUD  instruction, 
practice,  HUD  posttest,  REO  overview,  and  REO  posttest,  respectively. 

Results  and  Discussion 

Table  3  shows  the  correlation  matrix  for  dependent  variables  that  were  included 
in  the  MANOVA.  There  were  significant  differences  detected  among  treatment  groups, 
multivariate  F(6,38)  =  2.94,  p  <  .05.  Univariate  tests  showed  significant  differences 
among  treatment  groups  for  RT  on  the  HUD  and  REO  posttests.  The  adjusted  means 
for  posttest  RT  are  reported  in  Table  4.  Part-task  subjects  were  significantly  faster  (M  = 
11.8)  than  whole-task  subjects  (M=  13.4)  on  the  HUD  posttest,  £(1,43)  =  7.97,  p<  .01. 
Part-task  subjects  were  also  faster  (M=  12.4)  than  whole-task  subjects  (M=  13.6)  on 
the  REO  posttest,  F(1 ,43)  =  5.03,  p  <  .05. 

Although  part-task  subjects  performed  better  than  whole-task  subjects  overall,  a 
significant  interaction  effect  on  RT  on  the  HUD  posttest  was  also  detected,  F(1 ,43)  = 

1 2.65,  p  <  .001 .  The  interaction  indicates  that  part-task  subjects’  advantage  in  speed 
was  mediated  by  type  of  instructional  control.  Part-task  subjects  were  faster  under 
program  control  (M=  10.8)  than  under  learner  control  (M=  12.9),  whereas  whole-task 
subjects  were  faster  under  learner  control  (M=  12.4)  than  under  program  control  (JM  = 
14.4).  This  interaction  is  illustrated  in  Figure  4. 

Type  of  training  task  had  a  significant  effect  on  the  scores  received  for  each 
practice  exercise,  multivariate  £(4,40)  =  6.04,  p<  .001.  Part-task  subjects  met  criteria 
on  more  targets  in  the  first  exercise  (M  =  4.3,  &Q=  1.37)  and  in  the  second  exercise  (M 
=  4.4,  £1)  =  1.71)  than  whole-task  subjects  (M  =  2.7,  £_Q  =  1 .58  and  M  =  3.1 ,  £_Q  =  1 .1 0, 
respectively),  £(1,43)  =  15.63,  p<  .001  and  £(1,43)  =  10.10,  p<  .005,  respectively. 
Part-task  subjects  also  spent  less  time  on  the  feedback  screens  (M  =  27.5,  £D  =  7.62 
s)  than  whole-task  subjects  (M  =  35.0,  £D  =  1 2.24  s),  £(1 ,43)  =  21 .67,  p  <  .001 . 
However,  differences  in  practice  scores  and  time  on  feedback  were  most  likely  due  to 
differences  in  the  practice  tasks.  Portions  of  the  target-estimation  task  were  evidently 
easier  to  perform  separately  than  simultaneously.  Also,  part-task  feedback 
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Table  3.  Correlations  Among  Dependent  Variables 
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Note.  Time  is  reported  in  seconds.  Covariate  =  pretest  response  time;  LC  =  learner  control;  PC  =  program  control. 


(target  symbol  showing  either  correct  location  or  correct  heading)  took  less  time  to 
examine  than  whole-task  feedback  (both  location  and  heading). 

The  initial  analysis  of  training  time  on  different  parts  of  the  program  also 
revealed  significant  differences  among  treatment  groups,  multivariate  F(4,41)  =  19.23, 
£<  .001.  Table  5  shows  a  breakdown  of  the  average  number  of  minutes  subjects 
spent  on  the  HUD  lessons,  practice,  additional  instruction,  REO  overview,  and  the 
complete  lesson  (time  summed  across  the  four  parts). 

Type  of  training  task  produced  a  trade-off  in  the  amount  of  time  subjects  spent 
on  the  HUD  lesson  material  and  the  practice.  Part-task  subjects  spent  more  time  (M  = 
13.1  min)  than  whole-task  subjects  (M=  9-9)  on  the  lesson  material,  F(1 ,44)  =  13.88,  & 
<  .001 .  However,  they  spent  less  time  (M  =  7.0  min)  than  whole-task  subjects  (M  =  9.5) 
on  the  practice,  £(1 ,44)  =  25.10,  £<  .001 .  The  difference  for  time  on  practice  was 
clearly  due  to  differences  in  the  design  of  part-  and  whole-task  items. 
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Table  5.  Training  Time  (in  minutes)  by  Type  of  Control  and  Task 


Task 

Learner  Control 

Program  Control 

Total 

HUD  Lessons 

Part-Task 

M 

13.7 

12.6 

13.1 

SD 

3.56 

2.92 

3.23 

Whole-Task 

M 

10.2 

9.5 

9.9 

SD 

3.30 

2.21 

2.77 

Total 

M 

11.9 

11.1 

11.5 

SD 

3.79 

2.98 

3.40 

Practice 

Part-Task 

M 

6.9 

7.1 

7.0 

SD 

1.24 

1.43 

1.31 

Whole-Task 

M 

8.9 

10.0 

9.5 

SD 

2.05 

1.98 

2.05 

Total 

M 

7.9 

8.6 

8.2 

SD 

1.96 

2.25 

2.11 

Additional  Instruction 

Part-Task 

M 

8.3 

6.6 

7.5 

SD 

7.97 

7.92 

7.82 

Whole-Task 

M 

2.5 

10.2 

6.3 

SD 

4.06 

7.79 

7.25 

Total 

M 

5.4 

8.4 

6.9 

SD 

6.87 

7.91 

7.48 

REO  Overview 

Part-Task 

M 

2.8 

2.5 

2.7 

SD 

0.89 

1.30 

1.10 

Whole-Task 

M 

3.0 

3.2 

3.1 

SD 

1.14 

1.21 

1.15 

Total 

M 

2.9 

2.8 

2.9 

SD 

1.01 

1.27 

1.13 

Complete  Lesson 

Part-Task 

M 

31.7 

28.8 

30.2 

SD 

9.72 

8.77 

9.22 

Whole-Task 

M 

24.6 

32.9 

28.7 

SD 

7.80 

9.28 

9.40 

Total 

M 

28.1 

30.8 

29.5 

SD 

9.35 

9.12 

9.24 
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Most  of  the  practice  items  in  the  part-task  treatment  consisted  of  only  a  portion  of  the 
target-estimation  task,  while  whole-task  items  consisted  of  the  entire  criterion  task. 

The  causal  factors  for  differences  in  time  on  the  HUD  instruction  appear  to  be 
associated  with  the  lesson  sequence  and  structure.  Whole-task  instruction  was 
massed  into  one  lesson  that  was  delivered  prior  to  any  practice.  In  contrast,  the 
distribution  of  part-task  instruction  enabled  subjects  to  practice  after  each  quarter  of 
the  initial  HUD  instruction.  Whole-task  subjects  probably  became  impatient  and 
moved  through  the  instruction  more  quickly  to  find  out  what  the  task  was  like,  and  this 
may  have  reduced  their  ability  to  understand  and  retain  essential  information  about 
the  task.  Such  enroute  behavior  could  account  for  whole-task  subjects’  slower  RT  on 
the  posttests. 

A  significant  control  by  task  interaction  effect  on  additional  instruction  time  was 
also  detected,  £(1 ,44)  =  5.35,  £<  .05.  Whole-task  subjects  spent  far  more  time  on 
additional  instruction  under  program  control  (M=  10.2  min)  than  under  learner  control 
(M=  2.5),  whereas  part-task  subjects  spent  a  little  less  time  under  program  control  (M 
=  6.6)  than  under  learner  control  (M  =  8.3).  This  interaction  was  produced  by 
differences  in  the  number  of  segments  of  additional  instruction  that  were  presented  in 
each  treatment  group. 

Subjects  under  learner  control  chose  to  complete  24%  of  the  part-task 
segments  but  only  7%  of  the  whole-task  segments.  In  contrast,  subjects  under 
program  control  were  routed  to  only  13%  of  the  part-task  segments  compared  to  23% 
of  the  whole-task  segments.  Part-task  subjects  under  learner  control  chose  to 
complete  a  segment  of  additional  instruction  on  31%  of  the  occasions  that  their  score 
was  above  criterion,  while  whole-task  subjects  chose  additional  instruction  on  only  6% 
of  these  occasions.  Also,  part-task  subjects  bypassed  additional  instruction  on  only 
8%  of  the  occasions  that  their  score  was  below  criterion  compared  to  15%  of  these 
occasions  in  the  whole-task  condition.  Thus,  part-task  subjects  tended  to  choose  more 
additional  instruction  regardless  of  their  practice  score  and  the  advisement,  while 
whole-task  subjects  were  more  likely  to  bypass  additional  instruction  even  when  their 
scores  were  below  criteria  and  they  were  advised  to  complete  it. 

The  differential  use  of  learner-control  options  suggests  that  part-  and  whole-task 
subjects  did  not  experience  the  same  cognitive  or  affective  states  during  training. 

Unlike  the  whole-task  instruction  that  immediately  described  the  criterion  task  in  its 


17 


entirety,  part-task  subjects  did  not  receive  a  full-task  description  until  the  fourth  quarter 
of  the  HUD  instruction.  If  part-task  subjects  thought  the  additional  instruction 
contained  a  more  complete  description  of  the  criterion  task,  they  probably  chose  it  for 
this  reason  rather  than  to  improve  their  performance.  In  contrast,  the  program-control 
treatment  provided  additional  instruction  on  occasions  when  the  individual  needed  it 
most.  These  differences  suggest  that  part-task  subjects  were  better  able  to  effectively 
use  additional  instruction  when  it  was  presented  as  a  function  of  their  performance. 
Learners  may  not  be  able  to  effectively  use  learner-control  options  in  part-task 
instructional  simulations  because  they  lack  a  robust  understanding  of  the  task  during 
initial  training. 

Whole-task  subjects’  tendency  to  bypass  additional  instruction  appears  to  be 
related  to  affective  rather  than  cognitive  factors.  Having  received  poor  scores  during 
the  first  two  practice  activities,  whole-task  subjects  may  have  become  discouraged 
with  the  program  and  were  unwilling  to  spend  much  additional  time  on  instruction. 
Students  tend  to  bypass  more  review  and  remediation  options  in  a  learner-controlled 
lesson  when  they  do  poorly  on  instructional  material  (Carrier,  1984;  Carrier,  Davidson, 
&  Williams,  1985;  Gay,  1986).  Clark  (1984)  explains  that  learners  avoid  the  extra  effort 
associated  with  choosing  additional  support  options  when  they  expect  to  fail  anyway. 
Hicken  (1991)  found  that  undergraduate  subjects  bypassed  more  learner-control 
options  for  additional  instruction  during  the  most  difficult  parts  of  a  CBT  lesson.  The 
present  results  indicate  that  learners  under  whole-task  training  may  avoid  options  for 
instructional  support  due  to  the  level  of  difficulty  and  potential  frustration  associated 
with  whole-task  practice. 

Additional  instruction  appears  to  have  had  a  negative  effect  on  RT,  especially 
for  whole-task  subjects.  Whole-task  subjects  under  learner  control  spent  the  least 
amount  of  time  on  additional  instruction  but  estimated  targets  19%  faster  than  whole- 
task  subjects  under  program  control.  Whole-task  subjects  under  program  control 
received  over  three  times  as  many  segments  as  those  under  learner  control.  The 
segments  of  additional  instruction  increased  the  time  lapse  between  practice  activities 
and  probably  disrupted  subjects’  concentration  on  the  practice.  Munro,  Fehling,  and 
Towne  (1985)  found  that  program-controlled  learners,  whose  practice  was  interrupted 
by  feedback  messages  each  time  they  made  an  error,  performed  less  well  on  a 
complex  task  than  those  who  were  able  to  control  the  display  of  feedback  via  learner 
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control.  They  concluded  that  the  processing  demands  associated  with  complex  tasks 
calls  for  instruction  that  does  not  intrude  on  learners’  attention  during  practice  events. 
Unlike  part-task  methods,  whole-task  training  requires  learners  to  acquire  all  the 
essential  information  about  a  task  before  attempting  to  practice  it.  In  such  a  situation, 
learners’  processing  demands  and  their  susceptibility  to  interference  in  working 
memory  are  probably  at  the  highest  level. 

GENERAL  DISCUSSION 

Results  of  the  present  studies  have  several  implications  for  the  design  of 
instructional  simulations: 

1 .  Progressive  part-task  training  is  probably  more  effective  than  whole-task  training  for 
teaching  display-interpretation  skills. 

2.  Instructional  events  that  occur  simultaneously  with  practice  activities  can  produce 
negative  effects  on  the  acquisition  and  retention  of  complex  skills. 

3.  Learner  and  program  control  differentially  affect  performance  and  on-task  behavior 
in  part-  and  whole-task  training  programs. 

4.  Program  control  is  probably  more  effective  than  learner  control  if  instructional 
support  is  delivered  in  such  a  way  that  is  minimally  disruptive  to  learners’  focus  on 
practice  tasks. 

5.  Learners  may  be  especially  susceptible  to  frustration  during  the  initial  stages  of 
whole-task  practice,  and  this  may  reduce  their  motivation  to  expend  effort  on 
instructional  tasks. 

Practice  may  be  the  single  most  important  part  of  instructional  simulation,  but 
content  material  that  clearly  describes  learning  tasks  is  essential  for  effective  practice. 
Only  the  instructional  content  that  enables  learners  to  perform  a  specific  practice  task 
should  be  delivered  prior  to  practice  in  order  to  minimize  load  on  working  memory.  No 
additional  instructional  support  should  be  presented  unless  learners  are  unable  to 
improve  their  performance  during  practice.  The  need  for  more  instructional  support 
during  practice  may  correspond  to  plateaus  in  RT  and  accuracy.  Yet,  such  information 
can  only  be  identified  and  used  for  control  purposes  by  programs  that  are 
sophisticated  and  powerful  enough  to  collect,  record,  and  analyze  multiple  responses 
in  real  time  during  simulation  activities.  Such  programs  should  provide  instructional 
support  that  is  brief  and  least  intrusive  to  learners’  attention  on  practice  tasks. 
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The  present  study  involved  subjects  who  had  no  prior  knowledge  of  the  task  or 
the  environment  in  which  this  type  of  task  is  normally  performed.  Additional  research 
should  examine  the  effects  of  control  and  training  task  on  pilots  who  possess  more 
knowledge  and  skills  associated  with  the  interpretation  of  cockpit  displays. 
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